Today's software industry faces challenges in intelligently distributing work to different servers and the services that reside within them. Some of these challenges include a lack of insight of the actual amount of work to be performed, effective resource utilization, concurrency without duplication of work, abnormally high response times, and potential malfunctioning of the service due to CPU overshoot of RAM deficiency or middle tier data store waiting. Current load balancer operations are designed based on the number of incoming requests, a weighted percentage, or a least used algorithm.
In addition, though performance testing methods exist to assess the capability of servers to perform particular services, these are typically done in a non-production environment before getting into production mode. However, it can be challenging to match the performance testing environment to the production environment. For instance, the data that a service is dealing with may not match the production data resulting in unpredictability. Re-calibration of the production servers “on-the-fly” presents a further challenge.